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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

!)□ Responsive to communication(s) filed on . 



2a)S This action is FINAL. 2b)D This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) [3 Claim(s) 1-60 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5M Claim(s) 3.4.11-22.25-27.35-37 and 39-57 is/are allowed. 

6) [3 Claim(s) 1.2.5.6.8-10.23.24.28-34 and 38 is/are rejected. 

7) [3 Claim(s) 7. 58-60 is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) Q The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) Q The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath qr declaration is objected to by the Examiner. 

Priority under 35 U.S.C. §§119 and 120 

1 3) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a>n All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application), 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
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1) CD Notice of References Cited (PTO-892) 4) O Interview Summary (PTO-413) Paper No(s). . 

2) Li Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) [_J Notice of Informal Patent Application (PTO-152) 

3) Q Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



1. Applicant's amendment of 7/31/02 has been considered in preparing this 
office action. 



2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 



3. This application currently names joint inventors. In considering 
patentability of the claims under 35 U.S.C. 103(a), the examiner presumes that the 
subject matter of the various claims was commonly owned at the time any inventions 
covered therein were made absent any evidence to the contrary. Applicant is advised 
of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of 
each claim that was not commonly owned at the time a later invention was made in 
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order for the examiner to consider the applicability of 35 U.S.C. 103(c) and potential 35 
U.S.C. 1 02(e), (f) or (g) prior art under 35 U.S.C. 1 03(a). 

* 

4. Claims 1 , 2, 5, 6, 8-10, 23, 24, 28-34, and 38 stand rejected under 35 
U.S.C. 103(a) as being unpatentable over Hsu et al. 

Hsu et al. Rendered obvious claim 1 by teaching the claimed one or more 
computer-readable media having stored thereon a computer program that, when 
executed by one or more processors of the computer, causes the one or more 
processors to perform acts, at figure 2; identifying a plurality of viewing rays to be used 
to constructed a view image of a scene represented by a mosaic; checking whether 
each of the plurality of viewing rays coincides with at least a portion of a captured 
image; and for each viewing ray that coincides with at least a portion of a captured 
image, selecting the portion of the captured image; for each viewing ray that does not 
coincide with at least a portion of a captured image, generating an interpolated portion 
by interpolating between at least two portions of one or more captured images based on 
a constant distance to objects in the scene; and combining the selected and 
interpolated portions to generate the view image, at columns 18-19, and more 
specifically with the teaching of in the more realistic situation of finite sampling of the 
frames in time, it has been shown in the art that any arbitrary view of the scene between 
any two views may be created by linear interpolation of the flow vectors between two 
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consecutive images. This implies that even when the captured frames are coarsely 
sampled in time, an arbitrarily dense time sampling may be created by view 
interpolation. Once such a sampling is available, the creation of the mosaic is trivial as 
discussed above. Note that the densely sampled complete image frames need not be 
created but only the central slit of each frame. 

While Hsu et al. Teaches most features claimed, it is noted that viewing rays are 
not explicitly taught. However, Hsu et al. teaches that in order for the mosaic to be 
coherent, points in the mosaic must be in one-to-one correspondence with points in the 
scene. Accordingly, given a reference coordinate system on a surface to which source 
images will be warped and combined, it is necessary to determine the exact spatial 
mapping between points in the reference coordinate system and pixels of each image, 
which teaching would have been obvious to one of ordinary skill in the art at the time of 
the instant invention to consider as viewing rays. 

Claim 2 further requires one or more computer-readable media as recited in 
claim 1 , wherein each portion comprises a slit image. Hsu et al. Teaches this at column 
18 as discussed above for the rejection of claim 1 . 

Claim 5 further requires one or more computer-readable media as recited in 
claim 1 , wherein the constant distance is different for each of a plurality of different 
portions of the scene. Hsu et al. Teaches a spherical mosaic at columns 13-14 which 
satisfies this requirement. 
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Claim 6 further requires one or more computer-readable media as recited in 
claim 1, wherein the combining comprises placing the selected and interpolated portions 
in a side-by-side arrangement to generate the view image. Hsu et al. Teaches this at 
column 18. 

Claim 8 further requires one or more computer-readable media as recited in 
claim 1 , wherein the view image comprises a stereo image including a left eye view and 
a right eye view. Hsu et al. Teaches stereo images at column 1 7 with the teaching of : 
5. With the global three-dimensional view transformations computed, next, at step 910, 
the routine computes dense depth/parallax maps with respect to the reference 2-sphere 
using the methods described in K. J. Hanna et al, "Combining Stereo and Motion 
Analysis for Direct Estimation of Scene Structure", Intl. Conf. Computer Vision, Berlin, 
May 1993, pp. 353-365. 

6. Using the parameters and maps produced in the foregoing steps, the 
routine at step 912 generates a three dimensional spherical mosaic. 

The above method can be used within the system of FIGS. 2 and 3 to construct 
a "stereo spherical mosaic" given the imagery to construct two or more spherical 
mosaics as input (i.e., rather than use two dimensional processing in the iterative 
process, three dimensional processing is used). 
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Claim 9 further requires one or more computer-readable media as recited in 
claim 1 , wherein the scene has been previously captured using one or more cameras. 
Hsu et al. Teaches this at figure 2. 

Claim 10 further requires one or more computer-readable media as recited in 
claim 1 , wherein the scene is a computer-synthesized scene. Hsu et al. Teaches this at 
column 4. 

Claims 23, 24, 28-34, and 38 recite features similar to those previously 
addressed in the above rejections, and the claims are rejected under similar respective 
rationale. 

5. Claims 7 and 58-60 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

6. Claims 3, 4, 1 1-22, 25-27, 35-37, and 39-57 are allowed. 

7. Applicant's arguments filed 7/31/02 have been fully considered but they 
are not persuasive. 
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In the remarks, applicant argues that Hsu is directed to generating a mosaic, not 
constructing, based on a previously captured mosaic, and that Hsu does not disclose or 
suggest the construction of the view image based on the previously captured mosaic. 
However, Hsu teaches view interpolation at column 18. View interpolation requires 
having a plurality of views from which to generate the interpolated view, thus, those 
views correspond to previously captured views. Further, Hsu teaches three- 
dimensional mosaics which corresponds to the claimed view image. 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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9. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Almis R Jankus whose telephone number is 703- 
305-9795. The examiner can normally be reached on M-F, 8-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman can be reached on 703-305-9798. The fax phone 
numbers for the organization where this application or proceeding is assigned are 703- 
308-6606 for regular communications and 703-308-6606 for After Final 
communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
4700. 

AJ 

October 21 , 2002 ^ /? 




. ALMIS fl. JANKUS 
PRIMARY EXAMINER 



